Solving and interpreting binary classification problems in marketing with SVMs

Marketing problems often involve inary classification of customers into ``buyers'' versus ``non-buyers'' or ``prefers brand A'' versus ``prefers brand B''. These cases require binary classification models such as logistic regression, linear, andquadratic discriminant analysis. A promising recent technique forthe binary classification problem is the Support Vector Machine(Vapnik (1995)), which has achieved outstanding results in areas ranging from Bioinformatics to Finance. In this paper, we compare the performance of the Support Vector Machine against standard binary classification techniques on a marketing data set and elaborate on the interpretation of the obtained results.

Nonlinear support vector machines through iterative majorization and I-splines

To minimize the primal support vector machine (SVM) problem, wepropose to use iterative majorization. To do so, we propose to use it-erative majorization. To allow for nonlinearity of the predictors, we use(non)monotone spline transformations. An advantage over the usual ker-nel approach in the dual problem is that the variables can be easily inter-preted. We illustrate this with an example from the literature.iterative majorization;support vector machines;I-Splines

Instance-Based penalization techniques for classification

Several instance-based large-margin classi¯ers have recentlybeen put forward in the literature: Support Hyperplanes, Nearest ConvexHull classifier, and Soft Nearest Neighbor. We examine those techniquesfrom a common fit-versus-complexity framework and study the links be-tween them. Finally, we compare the performance of these techniquesvis-a-vis each other and other standard classification methods.

SVM-Maj: a majorization approach to linear support vector machines with different hinge errors

Support vector machines (SVM) are becoming increasingly popular for the prediction of a binary dependent variable. SVMs perform very well with respect to competing techniques. Often, the solution of an SVM is obtained by switching to the dual. In this paper, we stick to the primal support vector machine (SVM) problem, study its effective aspects, and propose varieties of convex loss functions such as the standard for SVM with the absolute hinge error as well as the quadratic hinge and the Huber hinge errors. We present an iterative majorization algorithm that minimizes each of the adaptations. In addition, we show that many of the features of an SVM are also obtained by an optimal scaling approach to regression. We illustrate this with an example from the literature and do a comparison of different methods on several empirical data sets.iterative majorization;I-splines;absolute hinge error;huber hinge error;optimal scaling;quadratic hinge error;support vector machines

Nearest convex hull classification

Consider the classification task of assigning a test object toone of two or more possible groups, or classes. An intuitive way to proceedis to assign the object to that class, to which the distance is minimal. Asa distance measure to a class, we propose here to use the distance to theconvex hull of that class. Hence the name Nearest Convex Hull (NCH)classification for the method. Convex-hull overlap is handled through theintroduction of slack variables and kernels. In spirit and computationallythe method is therefore close to the popular Support Vector Machine(SVM) classifier. Advantages of the NCH classifier are its robustnessto outliers, good regularization properties and relatively easy handlingof multi-class problems. We compare the performance of NCH againststate-of-art techniques and report promising results.

Classification with support hyperplanes

A new classification method is proposed, called Support Hy-perplanes (SHs). To solve the binary classification task, SHs consider theset of all hyperplanes that do not make classification mistakes, referredto as semi-consistent hyperplanes. A test object is classified using thatsemi-consistent hyperplane, which is farthest away from it. In this way, agood balance between goodness-of-fit and model complexity is achieved,where model complexity is proxied by the distance between a test objectand a semi-consistent hyperplane. This idea of complexity resembles theone imputed in the width of the so-called margin between two classes,which arises in the context of Support Vector Machine learning. Classoverlap can be handled via the introduction of kernels and/or slack vari-ables. The performance of SHs against standard classifiers is promisingon several widely-used empirical data sets.Kernel methods;large margin and instance-based classifiers

Estimating the market share attraction model using support vector regressions.

We propose to estimate the parameters of the Market Share Attraction Model (Cooper & Nakanishi, 1988; Fok & Franses, 2004) in a novel way by using a non-parametric technique for function estimation called Support Vector Regressions (SVR)(Vapnik, 1995; Smola, 1996). Traditionally, the parameters of the Market Share Attraction Model are estimated via a Maximum Likelihood (ML) procedure, assuming that the data are drawn from a conditional Gaussian distribution. However, if the distribution is unknown, ML estimation may seriously fail (Vapnik, 1982). One way to tackle this problem is to introduce a linear loss function over the errors and a penalty on the magnitude of model coefficients. This leads to qualities such as robustness to outliers and avoidance of the problem of over¯tting. This kind of estimation forms the basis of the SVR technique, which, as we will argue, makes it a good candidate for solving the Market Share Attraction Model. We test the SVR approach to predict (the evolution of) the market shares of 36 car brands simultaneously and report stronger results than when using a ML estimation procedure.

Essays on Some Recent Penalization Methods with Applications in Finance and Marketing

The subject of this PhD research is within the areas of Econometrics and Artificial Intelligence. More concretely, it deals with the tasks of statistical regression and classification analysis. New classification methods have been proposed, as well as new applications of established ones in the areas of Finance and Marketing. The bulk of this PhD research centers on extending standard methods that fall under the general term of loss-versus-penalty classification techniques. These techniques build on the premises that a model that uses a finite amount of available data to be trained on should neither be too complex nor too simple in order to possess a good forecasting ability. New proposed classification techniques in this area are Support Hyperplanes, Nearest Convex Hull classification and Soft Nearest Neighbor. Next to the new techniques, new applications of some standard loss-versus-penalty methods have been put forward. Specifically, these are the application of the so-called Support Vector Machines (SVMs) for classification and regression analysis to financial time series forecasting, solving the Market Share Attraction model and solving and interpreting binary classification tasks in Marketing. In addition, this research focuses on new efficient solutions to SVMs using the so-called majorization algorithm. This algorithm provides for the possibility to incorporate various so-called loss functions while solving general SVM-like methods

Instance-Based penalization techniques for classification

Several instance-based large-margin classi¯ers have recently been put forward in the literature: Support Hyperplanes, Nearest Convex Hull classifier, and Soft Nearest Neighbor. We examine those techniques from a common fit-versus-complexity framework and study the links be- tween them. Finally, we compare the performance of these techniques vis-a-vis each other and other standard classification methods